BRANCHING RULES FOR SYMMETRIC MACDONALD 
POLYNOMIALS AND sL BASIC HYPERGEOMETRIC SERIES 
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Abstract. A one-parameter generalisation Rx{X;b) of the symmetric Mac- 
donald polynomials and interpolations Macdonald polynomials is studied from 
the point of view of branching rules. We establish a Fieri formula, evalua- 
tion symmetry, principal specialisation formula and g-difference equation for 
Rx{X;b). We also prove a new multiple g-Gauss summation formula and 
several further results for sin basic hypergeometric series based on Rx{X;b). 



1. Introduction 

Let A be a partition, i.e., A = (Ai, A2, . . . ) is a weakly decreasing sequence of 
nonnegative integers such that | A| := Ai + A2 + • • • is finite. Let the length ^A) of A 
be the number of nonzero A^. For x = (xi, . . . , Xn) and 1{X) < n the Schur function 
sx{x) is defined as 

, . , , deti<,^j<„(x/ ) ^ deti<,j<„(a;/ ) 

^ ' ' deti<,,,<„(a;r^) A{x) 

where A(a;) := ni<j(^i ~ ^j) the Vandermonde product. If /(A) > n then 
sx{x) := 0. From its definition it is clear that sx{x) is a symmetric polynomial in 
X of homogeneous degree |A|, and that {s\\ < n} forms a basis of the ring of 
symmetric functions A„ ;= Z[a;i, . . . ,a;„]®". 

A classical result for Schur functions is the combinatorial formula 

T 

Here the sum is over all semi-standard Young tableau T of shape A, and x'^ is 
shorthand for the monomial ^" with fii the number of squares of the 

tableau filled with the number i. One of the remarkable facts of (|1.2p is that it 
actually yields a symmetric function. 

The conventional way to view a semi-standard Young tableau of shape A (and 
length at most n) as a filling of a Young diagram with the numbers 1, 2, . . . , n such 
that squares are strictly increasing along columns and weakly increasing along rows. 
Given two partitions (or Young diagrams) A, fi write /i =^ A if /i C A and A — /i is a 
horizontal strip, i.e., if the skew diagram X — fi contains at most one square in each 
column. Then an alternative viewpoint is to consider a Young tableau of shape A 
as a sequence of partitions 

(1.3) = A(°) ^ A(i) =^ • • ■ A(") ^ A, 



(1.2) sxix)=J2^- 
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where denotes the empty partition. For example, for n = 6 the tableau 
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may be encoded as 

(3) =^ (6, 2) =^ (6, 2) =^ (7, 3, 1) =^ (7, 6, 2, 1) =^ (8, 6, 3, 2, 1). 

The above description implies that a recursive formulation of the Schur functions, 
equivalent to the combinatorial formula (jl.2p . is given by the branching rule 



(1.4) 



sx{xi, 



Sp(a;i 



, Xn~l), 



subject to the initial condition sa(-) = (5a, o ■ 

If we let C A be a pair of partitions and define the skew Schur function Sa//j 
of a single variable z as 



if /i =^ A 
otherwise, 

then the branching rule for Schur functions takes the more familiar form 

(1.5) SA(a;i, . . . ,X„) = ^ Sx/fj,{Xn)Sf^{xi, . . .,Xn-l)- 

^CA 

The Macdonald polynomials P\{x) = P\{x;q,t) [11,12] are an important q,t- 
generalisation of the Schur functions, and the Pa for 1{X) < n form a basis of the 
ring A„.F :— A„ F, where F = Q{q,t). A classical result in the theory is that 
the Macdonald polynomials satisfy a combinatorial formula not unlike that of the 
Schur functions; 



Px{x)=Y,^T^ 



where '0T — i^riqit) G F is a function that admits an explicit combinatorial de- 
scription. Importantly, if T has no more than n rows it factorises as 



)/A('-i)> 



where, as before, = A'"' =<;••■=<; A*^"^ = A is the sequence of partitions representing 
T. Probably the simplest (albeit non-combinatorial) expression for ijjx/^ is [12, Page 
342] 



(1.6) 



n 



l<i<j<l{fj.) ■' '•' ' 

where /(a) = [at) / {aq) ^ with {a)oo = nj>o(l ~ "9*)- (Note that ipx/p. e F since 
/I =^ A.) It follows from the above that the Macdonald polynomials, like the Schur 
functions, can be described by a simple branching rule. Namely, 

(1.7) PA(a;i,---,a;„) = ^ xl,^"^' Va/^ ^^(^^i' • • • ' ^"-i)' 

/i=^A 
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subject to the initial condition Px{-) = (5a, o- Again we may define a single- variable 
skew polynomial Px/^{z) = Px/^{z;q,t) for /i C A 



(1-8) Px,u{z) 



otherwise 



to turn the branching formula for the Macdonald polynomials into 
(1-9) -Pa (2:1, . . . = ^ P\/t,{xn)Pp.{xi, . . . 

pCA 

In view of the above two examples of symmetric functions admitting a recursive 
description in the form of a branching formula, a natural question is 

Can one find more general branching-type formulas that lead to 
symmetric functions? 

To fully appreciate the question we should point out that it is not at all obvious 
that if one were to take (II. 4|) (or, equivalently, (II. 5p ) as the definition of the Schur 
functions or (|1.7|) (or (|1.9p ) as the definition of the Macdonald polynomials, that 
the polynomials in question are symmetric in x. 

Assuming throughout that \q\ < 1 let the (generalised) q-shifted factorials be 
defined as follows: 

00 

(1.10a) {bU = {b;q)^:^l[{l-bq'), 

i=0 

(1.10b) (^)^-(^^'^)^--(it' 

i(A) 

(l.lOc) ib)x^{b;q,t)x:=l[ibt'-')x., 

i=l 

and let (&i, . . . , bi)k = (6i)fc • • • {bi)k and (&i, . . . , bi)x = ibi)x ■ ■ ■ {bi)x- Then prob- 
ably the best-known example of a branching rule generalising (|1.9p and resulting 
in symmetric polynomials is 

(1.11) Ma(xi,...,X„) = L-i/ Px/^.{Xn)M^{xi,...,Xn~l). 

The Mx{x) — Mx{x;q,t) are the interpolation Macdonald polynomials of Knop, 
Okounkov and Sahi [5,14,15,23], and (jl.lip is [15, Proposition 5.3]. For comparison 
with [15], we have 

Mxixi, ...,Xn)^ i("-l)l^lp^*(tl-"xi, . . .,t-^Xn~l,Xn). 

From (|l.lip it is clear that the top- homogeneous component of Ma (a;) is the Mac- 
donald polynomial Px{x) so that {Mx\ 1{X) < n} forms an inhomogeneous basis of 

An,F. 

For A = (Ai, . . . , A„) and /i — (/ii, . . . , /i„_i) such that A — /i is a horizontal 
strip and such that Ai < m denote by m" — A and m"^^ — /i the partitions (m — 
An, . . . , TO — Ai) and (to — /x„_i, . . . , to — /ii). Note that (to" — A) — (to"~^ — /i) is 
again a horizontal strip. It follows from (|1.6p that 

J^(m"-A)/(m-i-^)(l/2; l/g, lA) = Z-'^Px/^{z- l/g, l/t) = Z-'^Px/M 
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for Ai < m. It also follows from (|1.6|) and (11.111) that 

Mm^{l/xi, l/xn, l/q, l/t) _ „„ 

M„„-.(lM,...,l/x„_i;l/g,lA) 

If we replace {x,q,t) i-> (l/x, 1/g, l/t) in (jl.lip and then change A i-^ (to — 
A„, . . . , TO — Ai) and fi (m — fin-i, . . . , to — /xi) the branching rule for the inter- 
polation Macdonald polynomials may thus be recast as 

^ ■ ' MmAl/xi,...,l/Xn;l/q,l/t) 

^ jq^-'^xjt)^ M^„^i_^(l/xi, . . . , 1/xn-i; l/q, l/t) 

il'-'^^nh '^^^"""^ M„„-.(l/xi,...,l/a;„_i;l/<z,l/t) 

In this paper we consider a generahsation Rx{x; h) = R\{x\ b; q, t) of the Macdonald 
polynomials and the Macdonald interpolation polynomials defined recursively by 
the branching rule 



(1.13) Rx{xi,. . . ,Xn]h) := 2^ —tt r Px/^,{Xn)Rt,{xi, . . . ,Xn-l]b). 

Our interest in the functions R\{x;b) is not merely that they provide another 
example of a class of symmetric functions defined by a simple branching formula. 
Indeed, it may be shown that the more general 

(1.14) i?A(xi, . . . , :r„; «, b) ^ i^n/a)x{bx /t), p (^^^ (^^^ 

is also symmetric. Moreover R\{x\ a, b) is a limiting case (reducing BC„ symmetry 
to &n symmetry and breaking ellipticity) of Rains' BC„ symmetric abelian inter- 
polation functions [19,21] so that R\{x\b) is not actually new. However, it turns 
out that the function R\{x; b) has a number of interesting properties, not shared by 
Rx{x; a, b) or the more general BC„ abelian functions. For example, Rxix; b) satis- 
fies a Fieri formula that not just implies the standard Fieri formulas for Macdonald 
polynomials, but also gives an sl„ generalisation of the famous g-Gauss summa- 
tion formula. Specifically, with Rx{x;b) a suitable normalisation of the functions 
Rx{x;b) defined in equation (15. 1|) . and X an arbitrary finite alphabet, we prove 
that 

C\l^l, , , , -r-r {cx/a,Cx/b)oa 



(1-15) Eh; MxRx{x-c)^\[ 



^ ■.abJ ' ' -^-^ {cx,cx/ab)c 



For X = {1} this simplifies to the well-known g-Gauss sum 

{a,b)k ( _ {c/a,c/b)c 

k=0 



(c, c/ab)c 



2. Freliminaries on Macdonald polynomials 

We begin with a remark about notation. If / is a symmetric function we will 
often write f{X) with X — {xi, . . . (and refer to X as an alphabet) instead 
of f{x) with X = {xi, . . . ,Xn), the latter notation being reserved for function that 
are not (a priori) symmetric. Following this notation we also use f{X + Y) where 
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X + Y denotes the (disjoint) union of the alphabets X and Y, and f{X + z) where 
X + z denotes the alphabet X with the single letter z added. 

In the following we review some of the basics of Macdonald polynomial theory, 
most of which can be found in [11, 12]. 

Let Tq^xi be the g-shift operator acting on the variable Xi'. 

Then the Macdonald polynomials P\{X) = P\{X;q,t) for X = {xi, . . . ,Xn} are 
the unique polynomial eigenfunctions of the Macdonald operator 



(2-1) ^"(c)-E(-)'^'^(")n?^n^-' 



/C[nl is/ ^ iel 



where [n] :— {1, . . . ,n}. Explicitly, 

n 

(2.2) D^{c)Px{X) - Px{X) J](l - cq^-r-'). 

For later reference we state the coefficient of of this equation separately; if 

(2-3) 



'=1 j/i 



then 

(2.4) DlPx{X) = Px{X) q^'t"-'- 

1=1 

For each square s = G in the (Young) diagram of a partition (i.e., for 

each i £ {1, . . . , 1{X)} and j G {1, . . . , A^}, the arm-length a(s), arm-colength a'(s), 
leg- length l{s) and leg-colength l'{s) are given by 

a{s) = A, -j, a'{s) = j - 1 

and 

where A' is the conjugate of A, obtained by reflecting the diagram of A in the main 
diagonal. Note that the generalised g-shifted factorial (|1.10cp can be expressed in 
terms of the colengths as 

sex 

With the above notation we define the further g-shifted factorials = (g, t) , 
CA = cx{q,t) and bx = bx{q,t) as 

c^:=[](l-g''(^)+4'(^)) and ca := [^(l - 9"^'^*'*''^') 
sex sex 

and 

bx — . 

Ca 

Then the Macdonald polynomials Qx{X) = Qx{X;q,t) are defined as 

Qx{X) bxPxiX). 
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We also need the skew Macdonald polynomials Px/^ and Q\/^ defined for /i C A 

by 

(2.5a) P^(X + Y)^Y. Px/,iY)PAX) 

(2.5b) QA(X + n-EQvM(^)QM(^)- 

Note that P\/o = P\ and Qa/o ~ Qa, and that Pa/a = Q\/\ = 1- To simplify some 
later equations it will be useful to extend the definitions of P\/ ^ and Q\/^ to all 
partition pairs A, /i by setting Pa//^ = Qx/fi = if ^ A. From (12. (12. 2|) and 
(12.51) it follows that for a a scalar, 



(2.6a) P^,^{aX)^a}^-^^\p^,^{X) 
(2.6b) gA/M(a^) = a'^"^'QA/p(^), 

where aX := {_ax\ x e X}. 

For subsequent purposes it will be convenient to also introduce normalised (skew) 
Macdonald polynomials Pa/^ and Qa/^i as 

(2.7a) Px,^{X)^e^^^--^'^'^''^Px,^{X) = rW-"(^')^Q;,/^(X) 

•^A '^^ 



(2.7b) Qa/m(^) = ^ Qa/m(^) - ^ Px,M)^ 



d.. 

where 



KA) 

n{\) ■.= Y^^\s)^Y.^^~\)\. 

sex i=i 

Note that no additional factors arise in the normalised form of 



PA(x + y) = ^p^/^(y)p^(x) 
QA(x + y) = 5]Qv^(y)Q^(x), 



and that Px/q — Pa and Qx/o = Qx- If we define the structure constants f^^ 
f^(9,0 by 

(2.8) P^{X)P,{X) = J2^^^,Px{X), 



then 



cX j.n{p,)+n{u)~n{X) _2x_ fX 



with the g, t-Littlewood-Richardson coefficients. 

Below we make use of some limited A-ring notation (see [6] for more details). 
Let pr be the rth power-sum symmetric function 

Pr{X) 

xeX 
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and let p\ := Yli>iPXi- Then the p\{X) form a basis of A„ where n = Given 
a symmetric function /(X) we define 



f 



1 - 1 



where (j)a,b is the evaluation homomorphism given by 



(2.9) 



1 



In particular, /[(I — — i)] = /(l,^, . . . , ^) is known as the principal 

specialisation, which we will also denote as /((O)), and — t)] /(I, t, i^, . . . ). 

From [7, Equation (6.24)] 



Q 



1 - 1 



which, by = 5\u, also implies that 



(2.10) 



1 - a 
1 - t 



(a) A and P> 



1 -a 



1 - t 



This last equation of yields the well-known principal specialisation formula 

P;,((0))=t2«(A)(!!)A. 

CACa 

The Cauchy identity for (skew) Macdonald polynomials is given by 

EPA/.(x)Q./.(r) = f n n ep^a(^)q.a(^)- 

A \xeXyeY ^''y>°° J A 

The product on the right-hand side may alternatively be expressed in terms of the 
power-sum symmetric functions as 

" 1 1 - r 



It thus follows from (j2.9p and ()2.10p . as well as some elementary manipulations, 
that application of (t>a,c (acting on Y) turns the Cauchy identity into 



(2.11) eqv^ pvmW= ( n 



xex 



{ax)c 



A 



a — c 



1 - t 



For fi ^ i> ~ (followed by the substitution X —> X/a and then a c/a) this is 
the g-binomial identity for Macdonald polynomials [4, 13] 

{ax)oo 



(2.12) 



E(a)APA(x) = n 



A xex 

For later reference we also state the more general (/i, t^) = (0,/j,) instance of (|2.1ip 



(2.13) 



(bx) 



xex 



{ax)r 



A 



a — b 



1 - t 



PxiX). 



For reasons outlined below we will refer to this as a Fieri formula. 
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Let ^a/m = </'a,m(9,0 and ^1/^ = V^a, be defined by 
t^V-a/a* and -0a,m — V^A'/m' 

Op 

(For combinatorial expressions for all of '0a//^:'0a/p and 0a//j: see [12]). Further 
let f;(^)(X) :— P(^j,){X) {t)r/{q)r and er(^) the rth elementary symmetric function. 
Then the Macdonald polynomials P\{X) satisfy the Fieri formulas 



(2.14a) 
(2.14b) 



P^{X)gAX)= ^x/^Px{X) 

A-p|=r 

P,{X)er{X)^ ^'x/,Px{X). 



Now observe that (|2.13p for b = at yields 

{atx) 



whereas for a — bq it yields 



p.w n ^ - e«""^'qv.(i)pa(x), 



P,iX)Y[{l~bx)=J2b^^-'^Q 



A/m 



g-1 
1 - 1 



Px{X). 



Since 



Y ^"-grix) = n and Y^-bY^M) = n (1 - 



[ax) 



r>Q 



r>0 

we therefore have 

P^{X)gr{X)= Y Qx/^{^)Px{X) 

1A-M|=r 

and 

'9-1 



1 - t 



P^{X)er{X) = {-ly J2 Qa/p 

A-p|=r 

where we have also used (j2.7bp . Identifying 

Va/ai if M =^ 
otherwise 
and 



Px{X), 



Qa/m(1) = 



(2.15) 



Qa/p 



[9-11 


■1 


1— ' 





otherwise 

these two formulas are equivalent to the Fieri rules of (|2.14p . 

The skew polynomials can be used to define generalised g-binomial coefficients 
[9, 10, 14] as 

(2.16) 



"A' 




'A' 


:= Qa/p 

q,t 


1 


./^. 






1 -i 
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In particular has [^] = if fj, X and 



(m)' 
(fc) 



k 



with on the right the classical q-binomial coefficients [™] 



If \i) '■— (Al, . . . , Xi — 1, Ai+i, 

{i-qy- 



,A,) then [9] 



This, together with [9, Theoreme 9, Bis 



(2.17) 

where 
(2.18) 



n 


■ A ■ 




A(i) 











provides a simple recursive method to compute the generalised g-binomial coeffi- 
cients. 



Lemma 2.1. Assume that l{X),l{fi) < n. Then 



(2.19a) 

and 

(2.19b) 



^m((0)) 



det 



Sa((0)) l<ij<n 



Xi + ji — i 



En 



ift^q 



ift^l. 



In the above X]tj+=;i denotes a sum over compositions u G N" in the (3„ orbit 

of /i. 

Proof. Assume that t — q. Then (|2.16p simplifies to 



(2.20) 



ri(M)-«(A) £A 



l-q 



where we have also used (j2.7bl) and the fact that for t = q the (skew) Macdonald 
polynomials reduce to the (skew) Schur functions. Let hr{X) be the rth complete 
symmetric function. By application of the Jacobi-Trudi identity [12, Equation 
(1.5.4)] 

■^a/m = iiet {hx,^^^-i+j) ioin>l{X), 

l<i,j<n 

and the principal specialisation formula [12, page 44] 



(2.21) 



^a((0)) = 



n 



\i-\-n—i 



the generalised g-binomial coefficient (I2.20p can be expressed as a determinant 



^m((0)) 



det 



sa((0}) ^<^,3<n\{q,q)^■+n-J 



-\-n—i 
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Since 



1 



1 

{<l)r 



this establishes the first claim. 

The second claim follows in analogous manner. Since making the substitution 
t = 1 in the right-hand side of (|2.16p is somewhat problematic it is best to first use 
the symmetry [14, Equation (2.12)] 



(2.22) 
Since 



"A 




'X'' 






q.t 




t-\q-^ 



P^/^{X;l,t)^ey^^,{X) 



we therefore get 



Using 



and 



"A' 




'A'' 






9-M 




1,9 



)-n{\') c\'{l,q) 



1-9 



ii+=/i i=l 



1 



1-9 



for n > 1{X), 



as well as ca'(<z, 1) = ni('Z)Ai; it follows that 

[A 



(9;'7)r 



Finally replacing q ^ 1/q yields the second claim. 

3. Symmetric functions and branching rules 



□ 



In this section we consider the question posed in the introduction: 

Can one find new(?) branching-type formulas, similar to (|1.4p . 
(|1.7p and (jl.lip . that lead to symmetric functions? 
Assume that fc is a fixed nonnegative integer, and let a = (ai, a2, . . . , a^) denote 
a finite sequence of parameters. Then we are looking for branching coefficients 
f\/ti.{z\a) such that 

(3.1a) fx{xi, . . . ,a;„;a) = ^ /A/^(a;„; a)/^(a;i, . . . 

subject to the initial condition 

(3.1b) /A(-;a)=5A,o 

defines a symmetric function. In the above a' — {a[, . . . ,a'j,) — g{a). Of course, 
p. lap for n = 1 combined with (j3.1bp implies that 

f\{z-a) = /A/o(2;;a). 

If one wishes to only consider symmetric functions with the standard property 

(3.2) /A(xi,...,x„;a) =0 if > n 
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then the additional condition 

/V^(z;a) = if l{X)-l{p)>l 

must be imposed. 

Because we assume the branching coefficients to be independent of n, it may 
perhaps seem we are excluding interesting classes of symmetric functions such as 
the interpolation Macdonald polynomials. As will be shown shortly, assuming n- 
independence is not actually a restriction, and (jl.lip may easily be recovered as a 
special case of p.lal) . 

Now let us assume that (13. lap yields a symmetric function fx{xi, . . . , a::„; a) for 
all n < N. (For N = and = 1 this is obviously not an assumption.) Then, 

f\{xi, . . . ,a;„_i,y,z;a) = ^ h/t,{z]a)f^,{xi, . . . ,Xn-i,y;a') 

= X! fx/i.{z;a)f^/^{y;a')f^{xi,...,Xn-i;&") 

is a symmetric function in xi, . . . , Xn-i, y (for n < N). For it to also be a symmetric 
function in xi, . . . , Xn-i, y, z we must have 

f\{xi, . . .,Xn-i,y,z;a.) = f\{xi, . . . ,Xn-i, z,y;a), 

implying that for fixed A 

X! h/ii{z\3)ff,/^,{y;B')fu{xi, . . . ,a;„_i;a") 

= X! fx/i^iy'^)ft^/i^iz;a.')f^{xi,...,Xn-i;a"), 

where a" .9 (a')- Hence a sufficient condition for (|3.1ap to yield a symmetric 
function is 

(3-3) f\/t^iz;a)ff,/^{y;a') = ^ /^/^(y; a)/^/^(z; a') 

z^C/^CA i/C/^CA 

for partitions A, v such that C A. 

As a first example let us show how to recover the Macdonald interpolation poly- 
nomials of the introduction. To this end we take a = (a), a' = (a/i), and 

f\/f,{z; a) = fx/^{z; a) ^ I^^Ma P\/t.{z)- 

Clearly, the resulting polynomials fx{x;a) correspond to the interpolation polyno- 
mials after the specialisation a = To see that (|3.3I) is indeed satisfied we 
substitute the above choice for the branching coefficient (recall the convention that 
Px/^ if /i ^ A) to obtain 

.A V- ia/z)xia/ty)f, {a/v)\{a/tz)^ 
^'■'^ ^ (a/z),(a/t,). ^V.(-)^./.(l/) = E ^a/yUaM. P^/MP.lM- 
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The identity ()3.4p is easily proved using Rains' Sears transformation for skew Mac- 
donald polynomials [20, Corollary 4.9] 



/o t:^ {aq/b,aq/c)x{d,e)f, 

{aq/d,aq/e)x{b, c)^ 
(aq/d, aq/e)f_,{b, c)^ 



1 — aq/de 



P, 



E 



p 



i-t 

1 — aq/bc 



aq/de — a^q^/bcde 



\-t 



l-t 

aq/bc — a?q^ /bcde 



After simultaneously replacing (a, b, c, d, e) i— > (c, a/tz, cqz/a, a/ty, cqy/a) and tak- 
ing the c — > oo limit we obtain ()3.4p . 

If, more generally, we let (a, 6, c, rf, e) i-^ (c, a/6z, cqz/a, a/by, cqy/a) in p.5|) and 
take the c — > oo limit we find that 



{a/z)\{a/by)^ 
{a/z),{a/by), ^'^ 



bz 



l-t 



y - by 



l-t 



Sr^ {a/y)x{a/bz)^, 
2^ ia/y),{a/bz), 



y-by 



l-t 



P 



bz 



1 - 1 



The Macdonald interpolation polynomials may thus be generalised by taking a 
(a, 6), a' = {a/b,b) and 



l-t 



Proposition 3.1. The polynomials Mx(xi, . . . , Xn', a, b) — AI\{xi, . . . , Xn', a, b; q, t) 
defined by 

'I - b] 

I U/ J^r, In 



Mxixi, 



^ (a/a:„)p 



l-t 



M^{xi, . . . ,Xn-i;a/b,b) 



subject to M\{-; a, b) — (5a, o o,re symmetric. Moreover, the interpolation Macdonald 
polynomials corresponds to 

Mx{xi, . . .,Xn) = Mx{xi, . . . ,a;„;t"~\t). 

The polynomials AIx{xi, . . . , Xn', a,b) are an example of a class of symmetric 
functions for which 1{X) > n does not imply vanishing. For example, 

'1 - b] 



Mx{z;a,b) = z\^\{a/z)xPx 



l-t 



^n(X)J\\ ia/z,b)x 

cx 



The next example corresponds to Okounkov's BC„ symmetric interpolation poly- 
nomials [16] (see also [17,20]). 

Proposition 3.2. If we take take a = (a, 6), a' = {a/t,b/t) and 
/A/^(z;a) = fx/f,{z;a,b) = j^y^^^ ^A/^CV^) 

in p. lap then the resulting functions fx{x;a,b) = fx{x; a,b; q,t) are symmetric. 

Writing Ox{x; a,b) instead of fx{x;a,b), the (Laurent) polynomials Ox{x; a,b) 
satisfy the symmetries 



O 



x{x; a, b) - (^) '^'0a(1/x; b, a) - (^) ^ O x{l / x; l/a, 1/6; l/q, l/t) 
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(This follows easily using that Px{X;l/q,l/t) = Px{X,q,t).) Moreover, Ok- 
ounkov's BC„ interpolation Macdonald polynomials P^{x; q,t, s) follow as 

(Since Ox{ax; a, b/a) — a^''^^Ox{x; 1, b) the Oxix; a, b) are not more general than the 
P^{x;q,t,s).) 

Proof of Proposition p.2|) . Substituting the claim in ()3.3|) and using ()2.6p gives 



^ {a/z,bz)x{a/yt,byt)^ 

^ ia/z,bz)^^ia/yt,byt). /-A/^UJ^-^/.W 



ia/y,by)xia/zt,bzt)^ 
^ ia/y,by),ia/zt,bzt), n/^lj^^/.l^J- 



This is p.Sp with (a, 6, c, d, e) i— > {ab/qt, bz/t, a/ zt, a/yt, by/t). □ 

Our final example will (in the limit) lead to the functions studied in the remainder 
of the paper. 

Proposition 3.3. // we take a = (a, b), a' = {at, b) and 

f I \ f I u\ {z/a)\{bz/t)^ 
/A/.(.;a) = fx/,{z;a,b) = ^^J^^^^P>^,M 

in p. lap then the resulting functions fx{x;a,b) — fx{x; a,b; q,t) are symmetric. 
Proof. Substituting the claim in (13. 3p and using (|2.6p gives 

{z/a,by)x{bz/t,y/at)f, 
^ (z/a,by),{bz/t,y/at), n/^Ui^M/^W 

_ V- iy/a,bz)xiby/t,z/at)f, 

^ {y/a,bz),{by/t,z/at), n/.Uj/^M/. 

This is p.5p with (a, b, c, d, e) i-^ {byz/aqt, by/t, z/at, bz/t, y/at). □ 

If we write Rx{x; a, b) instead of fx{x; a, b) the symmetric functions of Proposi- 
tion 13.31 correspond to the functions described by the branching rule (|1.14p of the 
introduction. As already mentioned there, the Rx{x;a,b) are not new, and follow 
as a special limiting case of much more general functions studied by Rains [19,21]. 
More specifically, Rains defined a family of abelian interpolation functions 

i?*^"' {x;a,b) = i?*'"^ {x;a,b;q,t;p), 

where x = {xi, . . . ,x„). The R'^^^\x) are BC„ symmetric and, apart from param- 
eters a,b,q,t, depend on an elliptic nome p. In [21, Theorem 4.16] Rains proved 
the branching rule 

Rl^"'~^^\xi, . ..,Xn+i;a,b) = ^ c'-^^ (xn+i; a,b)R*J''\xi, . . . , a;„; a, 6), 

where the branching coefficient c'"^^ (z; a, b) — c^^^ {z; a, b; q, t; p) is expressed in terms 
of the elliptic binomial coefficient (^)j ^ (see [19, Equation (4.2)]) as 



(3.6) c^xiiz-^a,b) = 
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If we define 

R\{x; a, b) = Rx{x; a, b; q, t) 



Pa((0)) lim i?t(")(pl/4^; p^/^at"-\p^^%/q; 1/q, 1/t; p) 



b / p-^o 

and compute the corresponding limit of p.6p we obtain the branching rule p.l4p 
with n ^ n + 1. 



4. The symmetric function R\{x;b) 
In the remainder of the paper we consider the symmetric function 



q 



. j-l—n , u 

n(x)^niX)^ _j p^(^o^) (n:^\X\) 

X lim limRl^"\p^^^X;p^/^at"~\p^/H/q;l/q,l/t;p) 

which, alternatively, is defined by the branching rule (11.131) . Because R\ is a limiting 
case of the abelian interpolation function i?^^"-* many properties of former follow 
by taking appropriate limits in the results of [19,21]. For example, it follows from 
[21, Proposition 3.9] that the R\{X; b) for X = {xi, . . . , a;„} satisfy a g-difference 
equation generalising (|2.2p . Specifically, with Dn{b,c) the generahsed Macdonald 
operator 

D^{b, c) = ^ (-l)l^l<('^') n 11(1 " bx,)l[ic ~ Wi-"x,)T,,.. 

IC[n] iGl ' ^ iel 

we have 

n 

(4.1) Dn{b, c)Rx{X; b) - Rx{X; bq) [](1 - cq^H^-'). 

1=1 

Below we will first prove a number of elementary properties of the functions 
R\{X]b) using only the branching rule (|1.13p . Like the previous result, most of 
these can also be obtained by taking appropriate limits in results of Rains for the 
abelian interpolation functions R*^^"'\X]a,b). Then we give several deeper results 
for R\{X;b) (such as Theorem 15.11 and Corollaries 15. 2[ [5^ and \6l2^ that, to the 
best of our knowledge, have no analogues for i?**"^(X;a,6) or Rx{X;a,b). First 
however we restate the branching rule (|1.13p in the equivalent form 

(4.2) Rx{X- b) = J2 Px/,iz)R,iY; b), 

where X — Y + z. 

When X = {z} we find from ([42]) that 

(4.3) ^(^)(^;^)-(4- 
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From this it is clear that R(k){cz;b) ~ c''R(^k){z;bc) and that in the c —> oo hmit 
R(k){cz\ b) is given by (-6)-'=g-(2). It also shows that 

R(k+i){z]b) = R(k){z]bq). 
All three statements easily generalise to arbitrary X. 
Lemma 4.1. For c a scalar, 

Rx{cX;b) = c\^\Rx{X]bc). 
Lemma 4.2. For c a scalar and n := \X\, 

lim RxicX; b) = ( — ) )r(^)PA((0)). 

Lemma 4.3. Let n := \X\ and A a partition such that 1{X) = n. Define /i := 
(Ai — 1, . . . , A„ — 1). Then 

Rx{X-b)^R^{X-bq) n Y~b^- 

x<£X 

This last result allows the definition of R\{X;b) to be extended to all weakly 
decreasing integer sequences A = (Ai, . . . , A„). 



Proof of Lemmas \4-lW4-3\ By (I4.3[) all three statements are obviously true for X a 
single-letter alphabet, and we proceed by induction on n, the cardinality of X. 
By 63), 

(4.4) RxicX; &) = ^ ^^^^ P,^,icz)R,icY; b). 

Using (|2.6p and the appropriate induction hypothesis this yields 

Rx{cX- b) = cl^l ^ ^^^^ Px,^{z) R^{Y- be) = cl^li?A(X; be), 

establishing the first lemma. 

Taking the c ^ oo limit on both sides of (|4.4p and then using induction we get 



lim i?A(cX;fe) = y(-6)l''l-l^l9"(^')-"(^')r(^)-"('^)-I^IPA/u(l) 1™ Ru{cY;b) 



A" 



b 

fl-n \X\ 



'Wf^WY^P,/^(t--^)P^(f^-\...,t,l) 



where the last equality follows from 

To prove the final lemma we consider (|4.4p with c = 1 and, in accordance with 
the conditions of Lemma 14.31 with A„ > 1. Since Px/v{o?j vanishes unless X — v is 
a horizontal strip this implies that f„-i > 1. The summand also vanishes if l{v) > 
n — 1 so that we we may assume that l{i') — n— 1. Defining rj = [vi — 1, ... , Vn-i — 1) 
and /i = (Ai — 1, . . . , A„ — 1) and using induction, as well as 



{bz/t), 1 {bzq/t),, 
{bz)x 1-bz {bzq)f. 



and Px/,,{z) = zP^/^{z), 
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we get 

{bzq/t)^ 



RM-M) n T 



X 



where in the final step we have used (|4.2p and X = Y + z. □ 
Proposition 4.4 (Principal specialisation). For A such that 1{X) < n, 

By Lemma 14.11 this may be stated slightly more generally as 

Proo/. Iterating prT3| using 

we obtain the generalised branching rule 

. . . , x™, . . . , t, 1; 6) = ^ (i^^^ ^A/M((0))i?A.(a;i, . . . , 2:™; 6), 

for /(A) < n + m. When m ~ this results in the claim. □ 
Proposition 4.5 (Evaluation symmetry). For A such that 1{X) < n set 

{X) = iq^H"-\...,q^-H,l). 

Then 

Rx{a{t^);b) ^ R^{a{X);b) 
Rx{a{0);b) R^{a{0);b)' 

Proof. We may view the evaluation symmetry as a rational function identity in b. 
Hence it suffices to give a proof for b = where m runs over all integers such 

that Ai,Aii < m. But (see ([TT^ and ^TJ^) 

R (X ^ M„,.._A(l/a:i, ■ ■ ■ , l/xn, 1/q, 1/t) 

''^ ^ Af™„(l/xi,...,l/a;„;l/g,lA) 

and 

M^.{l/X;l/q,l/t)= II x"'{x;l/q)m 

so that we need to prove that 

M^._A(a(M)) _ M„„,^(a(A)) jaqt^-^)^ {aq^-"'t"-^)x 
M™._A(a(0)) Af™._^(a(0)) (agt«-i)A (agi-™t»-i)^ ' 

Making the substitutions A m" — X, ^ ^ m" — /i and a i— > aq^"H^~''^ we get 
Ma(q/(m)) ^ M^(a/(A)) (g"'f'-Va)^ (^"-Va)A ™(|Ah|^| 
MA(ag-™ti-"(0)) A/^(ag-™<i-"{0)) (g™i"-Va)A (^""Va)/. 
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Finally, by the principal specialisation formula for the interpolation Macdonald 
polynomials [14], 

AfA(ag-"ti-"(0)) _ (g'"t"-Va)A _„,|a| 
MA(a/(0}) (t"-Va)A 

so that we end up with 

M^{a/{^x)) ^ M^ja/jX)) 

This is the known evaluation symmetry of the interpolation Macdonald polynomials 
[14, Section 2]. □ 

It is clear from (|4?2|) that Rx{X;0) = P\{X) with on the right a Macdonald 
polynomial. The Macdonald polynomials in turn generalise the Jack polynomials 
p|"^(X), since P^°'\x) — hiHg^i PxiX; q, q^^"). Combining the last two equations 
it thus follows that 

P|")(X) = hm i?A(^;0;g,qi/«). 

Curiously, there is an alternative path from Rx{X;b) to the Jack polynomials as 
follows. For X an alphabet let 

Proposition 4.6. We have 

Pt\x) = \imRx{X;l;q,q'/"). 

Proof. Let X = Y + z he a, finite alphabet. 

Replacing {b,t) (l,g^/") in (|4.2p and taking the q ^ 1 limit yields 

Ri"\x) = Y^ii z)i'^i-i^ip(;;(z)<)(y), 

where is a skew Jack polynomial and 



Ri"\x) := limRx{X;l:q,q'/''). 



q^l 

Using the homogeneity of P|/^ the above can be rewritten as 
Comparing this with 

pl")(x) = 5]pi;)(z)p(")(y) 

the proposition follows. □ 
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5. Cauchy, Fieri and Gauss formulas for R\{X;b) 

Probably our most important new results for Rx(X; b) are generalisation of the 
skew Cauchy identity (|2.1ip . the Fieri formula (|2.13p and the q-Gauss formula 
(fTTSl) . 

Before we get to these result we first need a few more definitions. First of all, in 
analogy with (12. 7p , we set 



(5.1) 

so that (14. 2p becomes 



Furthermore, we also define the skew functions R\/^{X;b) by 



(5.2) 
and 

In other words, 



(6z)a 

Rx/^iX + Y;b)=Y, Ra/m(^; b)R^/,{Y; b) 



and Ra/^(X;0)-Pa/p(X). 

Theorem 5.1 (Skew Cauchy-type identity). Let ab = cd and X a finite alphabet. 
Then 



(5-3) Ei^Qv. 

V (b/c)„ 



1 - t 



n 

xex 



Ra/m(^;&) 

{cx, dx)oi 



{ax, bx)c 



1 - t 



K/x{X\d). 



Note that for 6 = the theorem simplifies to (|2.1ip . We defer the proof of (|5.3p 
till the end of this section and first list a number of corollaries. 

Corollary 5.2 (Fieri formula). Let ab — cd and X a finite alphabet. Then 



(5.4) 



rm(^;^) n 



{cx, dx)c 
{ax, bx)c 



1 - 1 



Rx{X;b). 



This follows from the theorem by taking = and then replacing v by fj,. 

When b,d equation (|5.4p yields the Fieri formula (|2.13p for Macdonald 
polynomials. When b,c ~> and a 1 such that b/c = d equation (|5.4p yields 
(after replacing d^ b) 



(5.5) 



E 



(&)a 
(&)m 



Px{X)^RM;b) n 



{bx)c 

{x)cx 
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For /i = this is the g-binomial formula for Macdonald polynomials (|2.12|) , and for 
6 = it is Lassalle's [9] 

E[:!lpAW=p.(x)n 



The Jack polynomial limit of (|5.5p is of particular interest. To concisely state 
this we need some more notation. Let 



(a) 



:— lim 



or, alternatively [3,8,18], 



P'^\xi + l,...,Xn + l) 



E 



AV")pi")(xi,...,a;„) 



where n is any integer such that n > 1{X). Further let 

i>i 

with {b)k^b{b+l)---{b + k~l), 

c'xia) ■.^l[iais) + l + lis)/a) 

and 

Using all of the above, replacing {b,q,t) in (|5.5p by {q^ ,q,q^^°') and taking the 
(formal) limit g ^ 1 with the aid of Proposition 14.61 we arrive at the following 
identity. 

Corollary 5.3 (Binomial formula for Jack polynomials). For X a finite alphabet 



E 



(/3;«)a ^AV"' (a) 



(/3;a)^ VA* 



1 



Another special case of (|5.4p worth stating is the following multivariable exten- 
sion of the i(j)i summation [2, II. 5], which follows straightforwardly by taking the 
a, d ^ limit. 



(5.6) 



y-^|A^M|(V£)AQ 



0-1 
1 - 1 



Rx{x-b)=p^{x) n 



(ca:)c 
(fex)c 



This provides an expansion of the right-hand side different from (|2.13p . 

If we let = in Theorem l5.1[ use (|2.10p and then replace (a, 6, c) ^-^ {c/ab, c, c/a) 
we obtain 

^/ c \ |A-^I (a, 6)a ^ /V N TT {cx/a,cx/b)oo 



^\ab 



{a,b), 



ra/m(^;c)= n 



(cx, cx/ab)c 



For /X = we state this separately. 
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Corollary 5.4 (s[„ q-Gauss sum). For X a finite alphabet 



(5.7) 



xex 



{cx/a, cx/b)c 
(ex, cx/ab)o^ 



As mentioned in the introduction, for X = {1} this simplifies to the standard 
g-Gauss sum (|1.16p thanks to (|4.3p . More generaUy, if we principally specialise 
X = t^~"(0) = {l,t^^, . . . ,t^^"} and use (|4.5|) the sin g-Gauss sum simplifies to 
Kaneko's g-Gauss sum for Macdonald polynomials [4, Proposition 5.4] 

^\ ab J (c)a " l\ ict^-\ct^-yab)^- 



As another consequence of the theorem we obtain an explicit expression for 
the Taylor series of Rf^{X;b) in b. For /i C A let (l)A//i be defined as (1)a//j = 
limo^i(a)A/(a)^. That is 

(i)a/m= n i^-Q^'^'^t-'^^^). 

sG A — /I 



Corollary 5.5. We have 

(5.8) 

or, equivalently, 



R^(X;&) = ^6l^-^l(l)A/^ 



A-/i|=r 



Pa(X), 



Px{X). 



Proof. Replacing {c,X) ^-^ {a/b,bX) in Lemma 14.11 and expressing the resulting 
identity in terms of the normalised function Ra(-'^; b) (see (|5.ip ) we get 



(5.9) 



RA(aX;6)- (^)'^'RA(6X;a) 



Combined with (|5.5p this implies that 



R^(aX;fe) 



n 



(te) 



{abx)c 



Pa(X). 



The summand vanishes unless /i C A and so we may add this as a restriction in the 
sum over A. Then the limit a — > 1 limit may be taken without causing ambiguities, 
and the claim follows. □ 



Corollary 15.51 implies the following simple expressions for R\{X;b) when t ~ q 
(Schur-like case) or t = 1 (monomial- like case). 

Proposition 5.6. Let X ^ {xi, . . . ,Xn}. Then 



(5.10a) 

and 



Rx{X;b) 



det 



A{X) l<ij<n \ {bXi)x^-j + i 



ift^q 



(5.10b) 



RxiX;b)^ J2 ( n 



M+=A i 



^ {bxi)u. 



ift^l. 
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Proof. Since the two claims are proved in almost identical fashion we only present a 
proof of (jS.lOap . The only significant difference is that the omitted proof of (j5.10bp 
uses (j2.19b|) instead of (|2.19ap . 

Assume that t = q. Let = A — (0, 1, . . . , n — 1) and suppose that Vn > 0. Since 
for any fc > 0, one has the expansion 



„/e+n— 1 



{bx)k 



r=0 



,/e+n~ l+r 



k + r 
r 



the matrix [x 



i/j +n— 1 



/{hxi)^.) factorises into the product of rectangular matrices 

k-l 
r ~ k 



(x^ )i— l,...,n 



and 



r=0,l, 



r — k 



r=0,l,... 
k—y\ ,...,iyn 



According to Cauchy-Binet theorem, the determinant on the right-hand side of 
(|5.10ap factorises into a sum of products of minors of these two matrices. 
On the other hand, by (jl.ip and (|2.19ap . the expansion (|5.8D gives 



Rx{X;b)A{X) = J2 ^'^"^'(IWa , det 

^ — ^ l<'i,7< 



det 



[x: 



where we have also used (|2.2ip and (|5.ip . Using v instead of A, and ij = i.i — 
(0, 1, ... ,71 — 1), this becomes 

,--11 



i?A(X;&)A(X) = ^ det 



1 



det ix'l') 



which is precisely the Cauchy-Binet expansion. The restriction > is lifted 
using Lemma □ 

Recall that the Macdonald polynomials are the eigenfunctions of the operator 
D^, see (12.41) . Because Px{X;q,t) — Px{X;q^^,t^^) this can also be stated as 

ViPx{X)=ujxPx{X), 

where 2?^ := £>^(g~^;t~^) and cux is given in (|2.18p . 



A second consequence of Corollary 15.51 it a generalisation of this identity as 
follows. Let 



X 1 X 7 



and 



i=i 



bx, 

q 



bxj 

q 



so that P,\(0) ^ 
Theorem 5.7. We have 

Vl{b)Rx{X-b) ^ uJxRAX-b). 
Proof. Define the operator f„ as 

n 

£n ■= XjAj [x] t "'^)(T'g-l 2,. — l) 
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Combining the {q,t) i-^ {q ^) instance of [9, Proposition 9] with (|2.22|) gives 



£:„PA(x) = (i-9)5]g-^'t'-"(i)A(, 



)/A 



A 



Paw (X), 



where A^*) := (Ai, . . . , Ai_i, Aj + 1, A^+i, . . . , Aj). 
Since 

I?i(6)=2?i(0)-6q-if„ 

this implies that 

71 



A(i)/A 



A 



PawW. 



By Corollary [53] we can now compute the action of 'Dl^{b) on R\{X; b): 

[AI 



Pi(&)R^(X;&) = 5]6l^-''l(lK/, 

AD^ 
AD^ 



Pi(6)PA(X) 

u;aPa(^) 



[A^ 



"AW 




"A" 


A 







Pa(.)(^) 



E^"'"'(i)v. 

AD^i 



cc-aPaIA^) 



A 



Ar 



Pa(X), 



-(i-9)EE^""'''9"''*^""(i)v, 

1=1 ADp 

where we have also used that (l)A//j(l);i/iy = (1)a/i/ for C /i C A. Recalling the 
recursion (|2.17p . the sum over i on the right can be performed to give 

Vi{b)R,{X; b)^uj,J2 [^1 Pa(^)- 

Again using (|5.8p completes the proof. □ 

As a third and final application of Corollary 15.51 we derive a simple expression 
for Dn{b, c)P^{x), to be compared with (|2.2p (obtained for 6 = 0) or with (|4.ip . 

Proposition 5.8. We /laue 

Proof. First we use (j2.7bp and p.l5p . as well as the fact that for A — a vertical 
strip (i.e., \i - = 0, 1) 



Xi—fli Xi^f_Li 

to put the proposition in the form 



g^(i)v„na-V'."-), 



AD/i 



g-1 
1 - 1 



PA(:r). 
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Since M^j^ :— [^] is lower-triangular (with M^j^fj, = 1), it is invertible (for the explicit 
inverse see [14, page 540]). Since (1)a/^(1)p/i/ = (1)a/i/ for C ^ C A the above 
equation is thus equivalent to 



(5.11) Z?„(6,c)^6l^--I(l)^/, 



(cgr-i)A 



Q 



g-1 

1 - 1 



Pa(x) 



for fixed v. 

Taking {a,b,c,d) i— > (eg, c, 0,1) in [20, Corollary 4.9] in Rains' generalised q- 
Pfaff-Saalschiitz sum [20, Corollary 4.9] 



(5.12) 
yields 



- b 



1-t 



1-t 



{bUch 



1 -t 



E 



Q 



A/m 



q-1 



1 - 1 



This allows the sum over /i on the right of (|5.1ip to be carried out, leading to 



i?„(5,c)E6l'^-''l(l)^/, 



n(i-c,-r-^)jp,)i-Hgr^^ 



Pa(x) 



Y[il-cq^H-^))j2{bq)\'-''Kl)x/. 



ADi/ 



PA(a;). 



By Corollary 15 . 51 this is the same as (|4.ip . 



□ 



5.1. Proof of Theorem 15. li To prove the theorem we first prepare the following 
result. 

Proposition 5.9. For /i, v partitions 

'1 - 61 



V ^"-^^ P , 



1-t 



(a)^* (a);. {c/a,c/b)oo sr^ {c/bt)x 



{c/t)f^ {c/b)u ic,c/ab)c 



1 - 6 



1 - 1 



Note that for = this is 



E 



(Q,fc)A 
(c)a 



Pa/mI- 



(^\ _ {c/a,c/b)aa {a,b)f, 
abJ (c, c/a6)oo (c/t)^' 



which, for fi — 0, simplifies to the g-Gauss sum (|1.16p . 
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Proof of Proposition 1 5. PI Key is the Cauchy-type identity for skew Macdonald poly- 
nomials due to Rains [22, Corollary 3.8]: 



(5.13) |E 



.|A| 



(e,/)A 



1 - c 
1 - t 



i-d 



1 - 1 



d) {e/dj/d), 
/'cd\\M [e/cd, f/cd)\ 



{a,b)} 



1 - c 
1 - t 



/A 



l~d 



1 - t 



provided that the sum on the left terminates and the balancing condition abcdq — 
eft holds. The prefactor Z refers to the sum on the left for /i = = 0, i.e., to 



Z = 



(a, 6); 



(a,b,c, d)x 
(e,/)A 



1 -c 



1 - t 



1 - d 



1 - i 



E 



Since {t)\ vanishes if 1{X) > 1, and since 

, , ^ 1 



1 - i 



1-t 
1 

(a)fe 



if follows that for (6, c) = (g , i) 



N 

E 



(a, d, g ^)feg* _ (e/a, e/d) 



{q,e,adq^ ^/e)k {e,e/ad) 



N 



where the second equality follows from the g-Pfaff-Saalschiitz sum [2, Equation 
(11.12)]. 

To make the same (6, c) = {q ^ ,t) specialisation in the right-hand side of (|5.13|) 
we first replace the sum over A by A C (using the fact that Pi,/x{X) = if A ^ z^). 
Then 

ib)x 



Yl (1 



is well-defined for b 
simplifies to 



-N 



Pa/m(1)Qa/. 



It thus follows that for for (6, c) = {q^^ ,t) (|5T3l) 
1 - d] 



l~t 



(«,9-'^)p /gXl"^! ia,q-''). (e/a, e/d) 



tJ (e/tj/t), 



N 



{e/d,f/d)^ {e,e/ad)N 

'dt^W{e/dtJ/dt)x 



xc 



ql {a,q-^) 



P./a(1)Q 



l-d 
1 - t 



for adq^ ^ — ef. Eliminating /, taking the limit TV — >• oo and using (|2.6p this 
results in the claim with (6, c) i— > (d, e). □ 



We are now prepared to prove Theorem 15.1 



BRANCHING RULES 



25 



Proof. We proceed by induction on n \X\. For n ~ we get the tautology 



jb/c) 
ib/c). 



1 - 1 



a — c 



1 - t 



For n > 1 we compute the sum on the right-hand side of (|5.3p , assuming the formula 
is true for the alphabet Y. (Recall that X = Y + z.) Using the branching rule (|5.2|) 
and 



(5.14) Qxf, 
we obtain 



1-t 



PA/,(z)-al''-'^lQA/. 



1 — c/a 



a — c 



1-t 



Ra/^(^;&) 

1 — c/a 



{blc),{hz)x 



1 - t 



Px/-n{az)R.^/f,{Y]h). 



The sum over A can be transformed by Proposition l5.9l with (a, 6, c) i-^ {b/ c, c/a^ hz) 
and fj, t—)^ rj. As a result 



5^,,(X; a, 6, c) = 7^;°° ^ '7uT7JJ\^ 



{cz,dz)oo y-^ {b/c)^{d z/t)x 
(az,bz)ooj^ {b/c)\ 

{cz, dz)oo yr-^ {dz/t)x 



{az, hz)oo , (dz) 



l-t 

^u/\{z)Sf,AY,a,b, c), 



where once again we have used (j5.14p . and where d = ah/c. By the induction 
hypothesis, S^^\(Y,a,h^c) may be replaced by the right-hand side of (|5.3p with 
[X, A, v) ^ iY, rj, A), leading to 



<? fv.r.h.\ TJ {cx,dx)oo ^ {b/c)f,{dz/t)x ^ 
6^,,(X, a,b,c}- [[ 2^ _____ Q^/^ 



xex 



{ax,bx)ooj^ {b/c)n{dz)u 



1 - t 



One final application of (|5.2|) results in 



(ca;, dx)oa ST^ (b/c) 



xex 



{ax, bx)oc ^ {b/c)rj 



Q 



1 - i 



P,VA(2)RA/„(y;d). 



^ly/riiX; d), 



which is the right-hand side of (|5.3p with X i—> ij. 



□ 



6. Transformation formulas for s[„ basic hypergeometric series 

In this final section we prove a number of additional identities for basic hyper- 
geometric series involving the function Ra(-'^; b). For easy comparison with known 
results for one-variable basic hypergeometric series we define 



ai, . 
bi,. 



..,ar 

■ .,b7' 



z;X 



(oi, ■ • ■ , ar)x 



,6s- 



((_1)|A|^«(A'),-„(A)\ - ,|A|R,(X;6,), 
i)a ^ ^ 
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where X is a finite alphabet. There is some redundancy in the above definition since 
bv Lemnial4l1zl^lR),(X:b,) = Rx{zX;bs/z). Since Rx{X]b) vanishes if A) > \X\, 
and recalling R(j.)(z; 6) /{hz)k (see (|4.3|) ). it follows that 



Hk) 
ai, . 



;^;{i} 



E 

fc=0 



(ai, . . . , ar)k 



«!, ■ 
61,. 



,bs)k 



■ , ar 



s — r+l 



with on the right the standard notation for one- variable basic hypergeometric series 
(with dependence on the base q suppressed). The reader is warned that only in 
this degenerate case do the "lower-parameters" &i, . . . , 6^ enjoy full Gg symmetry. 
Using the above notation the q-Gauss sum (15. 7p may be stated as 



(6.1) 



2*1 



a,b c 
c ab 



n 



(cx/a, cx/b)c 
(ex, cx/ab)oc 



To generalise this result we first prove the following transformation formula. 
Theorem 6.1. For f = de/bc and 11 a partition, 



E 



ia,b)\ 



Qa/m 



f/a- cf/a 



(d/c), 



n 

xex 



1 - t 

{fx,ex/a)r 
{ex,fx/a)r 



Rx{X;e) 



E 



{a, d/c)x 
id)x 



Q 



e/a — cf/a 



1-t 



Ra(^;/). 



Note that the substitution (6, c, e, /) 1-^ {d/c, d/b, /, e) interchanges the left- and 
right-hand sides. 

Proof. We replace ^ ^ v and then rename the summation index A on the right as 
/I. By the Fieri formula of Theorem 15.21 (with (o, b, c) 1— > [bc/d, e, a)) the term 



{fx,ex/a)c 



xex 



{exjx/a)r_ 



on the right-hand side can be expanded as 



E 

A 



(a)A ^ 



f/a -e/a 



1 - t 



RA(X;e), 



resulting in 

A 



E 



id), 



{d/c). 



f/a- cf/a 



E 



1 - 1 

{a)x{d/c)f, 
id). 



Rx{X;e) 



Q 



f/a- e/c 
1 -t 



Q 



s/a - cf/a 



1 - t 



Rx{X;e) 



By (j5.12p with {a,b,c,d) i—>- {d/c,b,d,e/ab) the sum over /i can be carried out, 
completing the proof. □ 

For /i = Theorem 16.11 simplifies to a multiple analogue of the g-Kummer- 
Thomae- Whipple formula [2, Equation (IILIO)] (corresponding to the formula be- 
low when X — {1}). 
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Corollary 6.2 (s[„ q-Kummer-Thoniae-Whipple formula). For f — de/bc, 



(6.2) 



3*2 



a,b,c / 
d,e 'a 



X 



n 

xex 



{fx,ex/a)c 
{ex, fx/a)c 



3<J'2 



a,d/b,d/c e 
d,f 'a 



For d = c this reduces to the q-Gauss sum (|6.ip . 

If we let c, d ^ in Thcorem l6.1l such that d/c = bf /e and then replace (e, /) 
(c, az), we find 



{a,b)) 



Ra(^;c) 

{cx/a, azx)c 



n 



(ca;, za;) 



E 

A 



I^-mI (a, abz/c)\ 
{a,abz/c)^ 



id 



R\{X; az) 



For /i = this is a multiple analogue of Heine's 2</'i transformation [2, Equation 
(IIL2)]. 



Corollary 6.3 (s[„ Heine transformation). We have 



a, b 



■.z-X 



n 

xex 



(cx/a, azx)c 

{ex, Zx)oo 



2$1 



a,abz/e c 



X 



Acknowledgements. We thank Eric Rains for helpful comments on a preliminary 
draft of this paper. 



References 

1. G. E. Andrews, The Theory of Partitions, Encyclopedia of Mathematics and its Applications, 
Vol. 2, (Addison- Wesley, Reading, Massachusetts, 1976). 

2. G. Gasper and M. Rahman, Basic Hypergeometric Series, Encyclopedia of Mathematics and 
its Applications, Vol. 35, second edition, (Cambridge University Press, Cambridge, 2004). 

3. J. Kaneko, Selberg integrals and hypergeometric functions associated with Jack polynomials, 
SIAM J. Math. Anal. 24 (1993), 1086-1110. 

4. J. Kaneko, q-Selberg integrals and Macdonald polynomials, Ann. Sci. Ecole Norm. Sup. 29 
(1996), 583-637. 

5. F. Knop, Symmetric and non-symmetric quantum Capelli polynomials, Comment. Math. 
Helv. 72 (1997), 84-100. 

6. A. Lascoux, Symmetric Functions and Combinatorial Operators on Polynomials, CBMS Re- 
gional Conference Series in Mathematics 99, (AMS, Providance, Rhode Island, 2003). 

7. A. Lascoux, E. M. Rains and S. O. Warnaar, Nonsymmetric interpolation Macdonald polyno- 
mials and g[„ basic hypergeometric series. Transform. Groups, to appear; arXiv:0807.1351 

8. M. Lassalle, Une formule du bindme generalisee pour les polyndmes de Jack, Rendus Acad. 
Sci. Paris, Ser. I 310 (1990), 253-256. 

9. M. Lassalle, Coefficients binomiaux generalises et polyndmes de Macdonald, J. Funct. Anal. 
158 (1998), 289-324. 

10. M. Lassalle, Quelques valeurs prises par les polyndmes de Macdonald decales, Ann. Inst. 
Fourier 49 (1999), 543-561. 

11. I. G. Macdonald, A new class of symmetric functions, Sem. Lothar. Combin. B20a (1988), 
41pp. 

12. I. G. Macdonald, Symmetric functions and Hall polynomials, second edition, (Oxford Univer- 
sity Press, New York, 1995). 

13. I. G. Macdonald, Hypergeometric series H, unpublished manuscript. 

14. A. Okounkov, Binomial formula for Macdonald polynomials and applications, Math. Res. 
Lett. 4 (1997), 533-553. 

15. A. Okounkov, (Shifted) Macdonald polynomials: q-integral representation and combinatorial 
formula, Compositio Math. 112 (1998), 147-182. 



28 



ALAIN LASCOUX AND S. OLE WARNAAR 



16. A. Okounkov, BC-type interpolation Macdonald polynomials and binomial formula for Koorn- 
winder polynomials, Transform. Groups 3 (1998), 181-207. 

17. A. Okounkov, Combinatorial formula for Macdonald polynomials and generic Macdonald 
polynomials, Transform. Groups 8 (2003), 293-305. 

18. A. Okounkov and G. Olshanski, Shifted Jack polynomials, binomial formula, and applications. 
Math. Res. Lett. 4 (1997), 69-78. 

19. E. M. Rains, Transformations of elliptic hypergeometric integrals, Ann. of Math., to appear; 
arXiv:math.QA/0309252 

20. E. M. Rains, BCn-symmetric polynomials. Transform. Groups 10 (2005), 63-132. 

21. E. M. Rains, BCn-symmetric abelian functions, Duke Math. J. 135 (2006), 99-180. 

22. E. M. Rains, Elliptic Littlewood identities, arXiv:0806.0871 

23. S. Sahi, Interpolation, integrality, and a generalization of Macdonald' s polynomials, Internat. 
Math. Res. Notices (1996), 457-471. 

CNRS, Institut Gaspard Monge, Universite Paris-Est, Marne-La-Vallee, France 

School of Mathematics and Physics, The University of Queensland, Brisbane, QLD 
4072, Australia 



